Redesigned Visualization Architecture & Expanded Chart Capabilities

Summary

This PR introduces a ground-up redesign of Data Formulator's visualization engine and backend architecture, delivering a multi-backend semantic chart compiler, a tiered sandbox system for secure code execution, and cloud-native workspace storage — along with 30+ chart types across four rendering backends.

Highlights

1. agents-chart — A New Semantic Visualization Library

We replaced the previous monolithic chart assembly logic with agents-chart, a standalone, pure-TypeScript visualization library built around a key insight: semantic types as the contract between AI and rendering.

Instead of asking the LLM to produce detailed, low-level chart specs (which look good but break on every user edit), or using library defaults (which are editable but look bad), agents-chart takes a third path:

LLM outputs:    chart type + field assignments + semantic types  (~10-line JSON)
Compiler does:  sizing, zero-baseline, formatting, color schemes, mark templates
User edits:     swap fields, change chart type, add facets → compiler re-derives (no AI call)

Architecture:

Each backend implements the same assemble() interface: data in, native spec out. Users retain full control over aesthetic fine-tuning using each library's own API — no abstraction tax.

2. Massively Expanded Chart Type Library

New chart icon assets added for area, bump, candlestick, density, lollipop, pyramid, radar, rose, streamgraph, strip plot, and waterfall charts.

3. Smarter Chart Recommendation

A new recommendation engine (core/recommendation.ts) intelligently suggests chart types based on field semantics and adapts field assignments when switching between chart types. The UI has been split into a streamlined SimpleChartRecBox component (~1,100 lines) plus a ChartGallery (~930 lines) for browsing the full chart type catalog.

4. Tiered Sandbox System

Replaced the single sandbox implementation with a three-tier execution model via an abstract Sandbox base class:

5. Cloud-Native Workspace Storage

New Azure Blob Storage workspace backend (~670 lines) alongside the existing local filesystem workspace, selected via a factory pattern (workspace_factory.py). Enables multi-user cloud deployments with the same API surface — tables stored as parquet, sessions as JSON, metadata as YAML.

6. Security Hardening

• Sandbox audit hooks block dangerous operations (file writes, subprocess spawning)
• Session export strips sensitive fields (API keys, model configs) before persisting
• Auth improvements for multi-tenant scenarios

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│  Frontend (React + Redux + TypeScript)                          │
│  ┌──────────────┐  ┌──────────────┐  ┌────────────────────────┐ │
│  │ Encoding     │  │ Chart        │  │ agents-chart compiler  │ │
│  │ Shelf (DnD)  │  │ Rec Engine   │  │ core → vegalite/       │ │
│  │              │  │              │  │        echarts/        │ │
│  │              │  │              │  │        chartjs/gofish  │ │
│  └──────┬───────┘  └──────┬───────┘  └────────┬───────────────┘ │
│         └─────────────────┴───────────────────┘                 │
│                           │                                     │
│  ┌────────────────────────▼─────────────────────────────────┐   │
│  │  ChartRenderService (headless: vega compile → SVG/PNG)   │   │
│  └──────────────────────────────────────────────────────────┘   │
└─────────────────────────────┬───────────────────────────────────┘
                              │ /api/*
┌─────────────────────────────▼───────────────────────────────────┐
│  Backend (Flask + LiteLLM)                                      │
│  ┌──────────────┐  ┌──────────────┐  ┌────────────────────────┐ │
│  │ Agent Routes  │  │ Table Routes │  │ Session Routes         │ │
│  │ (10+ agents)  │  │ (DuckDB)    │  │ (save/load/export)     │ │
│  └──────┬───────┘  └──────────────┘  └────────────────────────┘ │
│         │                                                       │
│  ┌──────▼──────────────────┐  ┌────────────────────────────────┐│
│  │ Sandbox (local/docker)  │  │ Workspace (local/Azure Blob)   ││
│  └─────────────────────────┘  └────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────┘

Breaking Changes

• The monolithic py_sandbox.py has been replaced by the sandbox/ package with base.py, local_sandbox.py, docker_sandbox.py, and not_a_sandbox.py.
• Sandbox selection is now configured via CLI argument (--sandbox-type).